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ABSTRACT: Digital Image Inpainting is challenging and interesting research area, because one has to restore the area 
which is not visible but important to visually complete the image. This technique has found widespread use in applications 
such as restoration, error recovery, multimedia editing, and video privacy protection. Because of the strong human visual 
perception, a very effective technique is required for digital image inpainting. Most automatic techniques are 
computationally intensive and unable to repair large holes. Existing methods use interpolation methods where surrounding 
information is not adequate for image interpolation and chain codes for contour matching for small damaged area 
reconstruction. But, reconstructed image has not given up to the mark results. This paper proposed an effective inpainting 
technique in order to improve the inpainting result. The method proposed in this paper uses Run Length Coding for shape 
tracking along with CNN approach, because Run Length coding track the shape of an image, which is better than the several 
methods available for shape tracking. 
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extrapolate the gray value and gradient outside the region. 



I. Introduction 

Reconstruction of missing or damaged portions of 
images is an ancient practice used extensively in artwork 
restoration. This activity, also known as inpainting or 
retouching, consists of filling in the missing areas or 
modifying the damaged ones in a manner non-detectable 
by an observer not familiar with the original images. The 
goal of inpainting algorithms varies, depending on the 
application, from making the inpainted parts look 
consistent with the rest of the image, to making them as 
close as possible to the original image, restoration of 
photographs, films and paintings, to removal of 
occlusions, such as text, subtitles, stamps and 
advertisements from images. In addition, inpainting can 
also be used to produce special effects. While, 
traditionally skilled artists have performed image 
inpainting manually, currently digital techniques are used, 
e.g. for automatic restoration of scratched films. 



1.1 Digital Image Inpainting Techniques 

As a first step the user manually selects the portions of 
the image that will be restored. Then image restoration is 
done automatically, by filling these regions in with new 
information coming from the surrounding or cell in our 
case. In order to produce a perceptually plausible 
reconstruction, an inpainting technique must attempt to 
continue the isophotes (line of equal gray value) as 
smoothly as possible inside the reconstruction region. In 
other words the missing region should be inpainted so that 
inpainted gray value and gradient 



Several inpainting methods are based on the above ideas. 

Bertalmio et al. first introduced the notion of digital 
image inpainting and used third order partial differential 
equations (PDE) [4], [5], [9] to diffuse the known image 
information into the missing regions. Later, this 
inpainting approach was modified to take into account the 
direction of the level lines, called isophotes, and to relate 
it to the Navier-Stokes flow [6], [7, [21]. 

This operation propagates information into the masked 
region while preserving the edges. Anisotropic diffusion 
is used to preserve edges across the inpainted regions [13]. 
For further discussion of various methods, see the recent 
survey articles [10], [19]. The algorithms proposed in the 
literature differ depending on the assumptions made about 
the properties of the image. For example, the total 
variation (TV) inpainting model proposed, based on the 
Euler-Lagrange equation, employs anisotropic diffusion 
based on the contrast of the isophotes inside the 
inpainting domain [8]. This model, designed for 
inpainting small regions, does a good job at removing 
noise, but does not connect broken edges (single lines 
embedded in a uniform background). The Curvature- 
Driven Diffusion (CDD) model, extends the TV algorithm 
to also take into account geometric information of 
isophotes when defining the 'strength' of the diffusion 
process, thus allowing the inpainting to proceed over 
larger areas. Although some of the broken edges are 
connected by the CDD approach, the resulting criteria for 
stopping the inpainting, the process is constantly applied 
to all masked pixels, regardless of the local smoothness of 
the region. As a result, computationally expensive 
operations might be unnecessarily performed, resulting in 
lengthy processing time. Thus, although non-linear PDE- 
based image restoration methods have the potential of 
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systematically preserving edges, fast numerical 
implementations are difficult to design. 

II. Contour ideas using run length code 

Contour tracing is one of many preprocessing 
techniques performed on digital images in order to extract 
information about their general shape. Once the contour 
of a given pattern is extracted, its different characteristics 
will be examined and used as features which will later on 
be used in pattern classification. Therefore, correct 
extraction of the contour will produce more accurate 
features which will increase the chances of correctly 
classifying a given pattern. The contour pixels are 
generally a small subset of the total number of pixels 
representing a pattern. Therefore, the amount of 
computation is greatly reduced when we run feature 
extracting algorithms on the contour instead of on the 
whole pattern. Since the contour shares a lot of features 
with the original pattern, the feature extraction process 
becomes much more efficient when performed on the 
contour rather on the original pattern. In 
conclusion, contour tracing is often a major contributor 
to the efficiency of the feature extraction process an 
essential process in the field of pattern recognition. For 
images retrieval, low-level visual features are color, 
texture, and shape. Among these features, shape is the 
most important because it represents significant regions or 
relevant objects in images. Extensive work has been done 
in shape retrieval. 

In general, shape representations are classified into two 
categories: boundary-based and region-based. The first 
one describes the considered region by using its external 
characteristics (i.e. the pixels along the object boundary) 
while the second one represents the considered region by 
using its internal characteristics (i.e. the pixels contained 
in the region). Several shape description approaches have 
been developed in the two categories. For example, area, 
compactness, is bounding box for the region-based 
category, and perimeter, curvature for the boundary -based 
category. Several techniques such as chain code, crack 
code and run length code etc. are existing to represent the 
region or object by describing its contour. 

2.1 RUN LENGTH CODING 

Run length coding used in [20] is a simple yet useful 
technique in lossless coding. Run Length Coding (RLC) 
is a conceptually simple form of compression. RLC 
consists of the process of searching for repeated runs of a 
single symbol in an input stream, and replacing them by a 
single instance of the symbol and a run count. Run-length 
coding is performed in order to extract the continuous flat 
parts from the contour. With run length code and 
thresholding, we can get the lengths of the flat segments 
of the contour, which are used for shape comparisons. 
From the flat segment lengths, we can construct a vector. 
The length of the vectors may not be equal. To compare 
vectors with different lengths, we use down sample to 
make them have the same length. To compare the down- 



sampled vectors, circular shifts are performed. Finally the 
minimal standard deviation of the ratio of the shifted 
vectors is used for the measurement of the shapes. 




Fig-1 Experimental result of shape tracking 



2.2 Cellular Neural Network 

Cellular Neural Networks (CNN) is analog, continuous 
time, nonlinear dynamic systems and formally belongs to 
the class of recurrent Neural Networks. Since their 
introduction in 1988 by Chua and Yang [3], they have 
been the subjects of intense research. The Cellular Neural 
Network (CNN) is an artificial neural network of the 
nearest neighbor interaction type. It has been widely used 
for image processing, pattern recognition, moving object 
detection, target classification, signal processing, 
augmented reality and solving partial differential 
equations etc. The Cellular Neural Network 
Complementary Metal Oxide Semiconductor CMOS 
array was implemented by Anguita. The design of a 
Cellular Neural Network template is an important 
problem, and has received wide attention in the recent 
years. This paper reports an efficient Algorithm exploiting 
the latency properties of Cellular Neural Networks along 
with popular numerical approximation algorithms. The 
dynamic equation of a cell C(i, j) in an M x N Cellular 
Neural Network is given by Chua and Yang [3]. 

III. The Rconstruction of Damaged Image Using 
CNN Method 

P.Elango a and K.Murugesan [2] proposed method 
using CNN Model, In which this paper is going to 
perform some modification with the belief of 
improvement. Considering an image or a video frame as a 
large Damaged Image Block (DIB) is placed in the CNN 
as shown in Fig. 2. Each DIB cell can be further 
subdivided into different Image Blocks (IBs), each of 
which may or may not contain damaged pixels. 
Furthermore, each Image Block is subdivided into several 
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Pixel Blocks (PBs) which are elementary objects to be 
inpainted. The recursive algorithm is shown in Algorithm- 
1 

The basic assumption is that color difference is a strong 
indication of the degree of details in an Image Block. The 
threshold value alpha (a) sets the criterion on whether a 
recursive call to the CNN based inpainting algorithm is 
required (the algorithm terminates when the DIB is small). 
In our implementation, the value of a is a percentage of 
color variance of an IB (i.e., the maximum var(IB) is 100). 
If the color variance of IB is greater than the threshold 
value a, the algorithm is called recursively to handle the 
next level of details. Otherwise, the algorithm further 
divides an IB into several pixel blocks in the CNN cell 
(i.e., PBs). Another criterion is the percentage of damaged 
pixels. 




Fig-2 Damaged Image Block (DIB), Image Block(IB) and Pixel Block 
(PB) with CNN cell 



We argue that if the percentage is too high, using 
surrounding color information to fix a pixel is less 
realistic as compared to using a global average color. In 
some severe cases, it is very difficult to use neighbouring 
cell colors. Note that, both thresholds are adjustable for 
the sake of analysis. The algorithm iterates through each 
of the PBs in an IB. If the percentage of damaged pixels 
in a PB is too high (i.e., greater than 02), the mean IB 
color is used [i.e., Mean Color (IB)]. One example is that 
the entire PB is damaged, so we must use the mean IB 
color. The function Damage Percent (PB) simply counts 
the number of damaged pixels in a PB. And, the function 
Fill (PB, C) takes a Pixel Block and a color C, and fills 
the Pixel Block with the color. Alternatively, if the 
percentage is still high (i.e., greater than pi), the mean PB 
color is used. 



Algorithm: 

Iupaiiitixig ( Damaeedlmaae Block( DIB) ) 

{ 

Damaged Image BJock = £ 
or 

Foi All(IB) i (DIB). (var(IB) > a then Lipauitu^( D3 ) 
or 

For all ( Pl\cI BJock ) £ LnageBlock 
If ( ( DainagedPeicent ( PixelBlock ) > 

TheiiFill( PrxelBlock.MeauColor( linage Bkk) ) 
Elself ( p, > = DamagedPercent ( Pixel Block ) > \ ) then 

Fill ( Pixel Block, Mean Color ( Pixel Block )) 

Else Find Contour Matdiiiig for damaged Portionand check Same matching 
within die image anywhere 

If I PattemFonnd i thenFilli Pixel Block damaged image withmeaii color). Found 
Pattern)) 

Algorithm- 1 Proposed Algorithm for CNN based 
Inpainting Method. 

Suppose a portion of an image or more number of cells 
is damaged and surrounding information not easy to get to 
inpaint or pi and P2 which are too greater than threshold 
value, one can find the contour matching (chain code) 
values of the damaged cells and check with damaged 
pattern found in the same image. If the contour matching 
values of damaged image and the pattern found in the 
surrounding are same, the damaged portion is to be 
inpainted using mean color of the pixels of the pattern 
found in the surrounding anywhere else. Note that, the 
computation of mean colors does not take damaged pixels 
into account. If the percentage is low enough (i.e., less 
than pi), neighbour pixels and cells are used for 
interpolation. The function Interpolate (PB) implemented 
in our algorithm uses a bi-linear interpolation technique. 
We further take an optimization step to improve the 
algorithm. When the filling function is called, we add 
noise on the boundaries of Pixel Blocks. Our proposed 
CNN based image inpainting algorithm is called again to 
remove these bounding boxes. Thus, block effect is 
reduced. There are three thresholds in the above algorithm, 
a, pi and P2 also used various combinations. The 
selection of P2 is aimed at testing the usage of mean color. 
Unless a pixel block is completely damaged, the mean 
color should be used. Thus, the selection of P2 should be 
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high. Since pi< |32, we select the values of pi accordingly. 
The threshold a is used to check the color variance. We 
try to cover a wide spectrum. The combinations of the 
above thresholds are all tested using more than 1000 
pictures. The values of a, pi and P2 have great impact on 
the outcome. In general, if a is less than 75, the average 
PSNR values of repaired pictures with respect to other 
parameters are stable. So we use a =80 in our 
implementation of an automatic inpainting tool. We chose 
P2=95 through our analysis. This means that unless the 
percentage of damaged pixels in a pixel block is higher 
than 95, the mean color of an outside big block should not 
be used. The value of pi is critical. If pi is less than 60, 
the result is not as good as expected. However, the PSNR 
values of the fixed pictures become stable when the value 
of pi becomes 80, 85, or 90. Conclusively, we choose 
a =80, pi=85, and P2=95. 

IV. Expected Result 

Proposed method will improve the visual quality of 
image and it will be able to restore the images with high 
noise ratio, because cellular neural network proved to be 
very useful regarding real-time image processing. Our 
objective is not to detect spikes or long vertical lines 
commonly reported in the literature. We aim to develop a 
technique to recover images with very high percentage of 
noise and achieve visually good output. 

The key factor of proposed method is no. of damaged 
pixels, color variance and shape tracking. When dealing 
with a region or object, several compact representations is 
available that can facilitate manipulation of and 
measurements on the object. This paper extends the 
previous approaches with the use of shape tracking 
method that has been used by researchers in biological 
vision. Researchers in biological vision have long 
hypothesized that image contours (ordered sets of edge 
pixels, or contour points) are a compact yet descriptive 
representation of object shape. In computer vision, there 
has been substantial interest in extracting contours from 
images as well as using object models based on contours 
for object recognition, and 3D image interpretation. 

V. Conclusion 

After surveying different methods, it is observed that, 
Considerable computing power is necessary to solve the 
image processing task described by variational computing. 
The findings of our perceptual evaluation of inpainting 
techniques yielded valuable information related to the 
quality of the inpainting. Combining this information with 
low level and mid level image features we are currently 
involved in introducing an objective computational model 
for automatically predicting the perceptual image 
inpainting quality. 

For the time being serial processing does not provide us 
with methods implementable in real-time. The cellular 
neural network proved to be very useful regarding real- 
time image processing. The reduction of computing time, 



due to parallel processing, can be obtained only if the 
processing algorithm can be implemented on CNN-UC. 

Even if variational methods are used, the determination 
of templates ensuring the gray-scale image and the 
desired processing remains a difficult problem, since the 
fact that the actually existing CNN chip can use only 
linear templates having 3*3 dimension has to be taken 
into consideration. In some cases templates satisfy these 
conditions by using nonlinear templates. Effective CNN 
implementation is still possible in CNN algorithms. 

Open Problems 

In this paper, we have surveyed all the recent 
inpainting techniques, based on that we deduced a model 
for combining Cellular Neural Network Ideas with 
different images. The classical CNN approach for Digital 
Image Inpainting framework has proven to be very 
effective in designing and improving image inpainting. 
We have explained that the fundamental techniques for 
inpaintings such as Isophotes (line of equal gray value), 
PDE, Navier - Stokes equation, Total Variation (TV) 
methods, and Curvature Driven Diffusion (CCD) ideas in 
the Literature Survey. As a result, it increases the 
execution time due to performing computationally 
expensive operations again and again. Since the serial 
processing methods are not suitable for real-time 
operations of image, our proposed CNN technique is the 
best one for real time image inpainting due to its parallel 
computation. Finally we post two interesting open 
problems. 

(i) Video inpainting- Video inpainting is a crucial and 
challenging area of image processing. It has profound 
applications in the movie industry, surveillance analysis, 
and dynamic vision analysis. The first open problem is: 
How to calculate and analyze the Peak Signal Noise Ratio 
(PSNR) values for video images with Cellular Neural 
Network as tool. 

(ii) Digital realization- Throughout our problem 
formulation, we found Numerical PDE has been a core 
computational tool for all the geometric inpainting models. 
The second open problem concerns fast and efficient 
digital implementation of the associated PDE's, especially 
for the high order ones and also we can utilize Cellular 
Neural Network Techniques for efficient digital 
realization. 
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